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Abstract — In this paper, we show how to construct a fac- 
tor graph from a network code. This provides a systematic 
framework for decoding using message passing algorithms. The 
proposed message passing decoder exploits knowledge of the 
underlying communications network topology to simplify decod- 
ing. For uniquely decodeable linear network codes on networks 
with error-free links, only the message supports (rather than 
the message values themselves) are required to be passed. This 
proposed simplified support message algorithm is an instance 
of the sum-product algorithm. Our message-passing framework 
provides a basis for the design of network codes and control 
of network topology with a view toward quantifiable complexity 
reduction in the sink terminals. 

I. Introduction 

Network coding [1] is a generalization of routing where 
intermediate nodes forward coded combinations of received 
packets (rather than simply switching). This approach yields 
many advantages, which by now are well documented in the 
literature [2]-[4]. For the single session multicast problem, 
it is well known that linear network codes are optimal and 
can achieve the fundamental limit characterized by the min- 
cut bound [5], Linear network codes are strongly motivated 
by practical considerations, namely the implementation of 
encoders and decoders. Encoders compute linear combinations 
of received packets, and decoding may be achieved by solving 
a system of linear equations (since the sink terminals receive 
a linear transformation of the source data). The standard 
approach for decoding is Gaussian elimination followed by 
back-substitution. The resulting decoding complexity is cubic 
in the size of the linear system, and is essentially independent 
of the underlying topology. 

However, the topology of the underlying communications 
network has a direct impact on the structure of the linear 
system that requires solution. This admits the possibility of 
faster decoding algorithms which exploit such structure. As 
one example, it is well known that band-diagonal systems 
may be solved with complexity that scales quadratically in 
the bandwidth [6]. Similarly, there are many low-complexity 
iterative solvers for large sparse linear systems [7] (although 
it should be noted that such iterative methods are usually 
confined to operating over real or complex fields). This paper 
is motivated by the possibility of faster decoding algorithms, 
which exploit knowledge of the network topology. 

Let a directed acyclic graph, G = (AT, C) model a com- 
munications network with nodes JV and directed links C. We 
assume that there are \K\ sources, generating source symbols 



Y s ,s & K. at nodes a(s) eJV. Let Yj be the (network coded) 
symbol transmitted along link I G C. 
For any link I £ C, define 

in(0 4 

{e e C : hcad(e) = tail(Z)} U {s e K : a(s) = tail(Z)} . 

To simplify notation, we will use the following conventions: 
(1) for any undirected graph n(v) is the set of neighboring 
nodes of v, (2) for any function g(x), we denote its support 
by A (g) = {x : g(x) ^ 0} and (3) for any function g(x, y) and 
for each y, we denote the set A (g(x\y)) = {x : g(x, y) ^ 0}. 

We are particularly interested in vector linear network codes 
over a finite field F, where y Sl yi S F™ for s € K, and I E C. 
Encoding is according to 

Vl = ^2 C ^2/e, I € C 
e£in(i) 

where the cj e are the local encoding coefficients and it 
is assumed that y s ,s S K. are independent and uniformly 
distributed over F". 

The following example motivates careful exploitation of 
network structure by the decoding algorithm. 

Example 1: Consider the network in Figure [TjAdth sources 
yi,...,yx- The receiver t aims to reconstruct all source 
symbols. Clearly, decoding can be done by directly solving 
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where the K x K matrix A depends on the choice of local 
encoding coefficients. Direct solution involves inversion of 
the K x K matrix A, which is 0(K 3 ). On the other hand, 
due to the network topology, it is easy to show that A = 
Bk-i • • • B2B1 where each matrix Bi has at most K + 2 non- 
zero entries, located on the diagonals and at the (i, i+ 1) and 
+ entries. Inversion of the Bi is very simple (essentially 
having the same complexity as inverting a 2 x 2 matrix). 
Therefore, we can solve the system of linear equations by 



computing 



Vl 

1)2 
VK 



b; 



B 



K-l 



Vl-k+i 

UL-K+2 
VL 



(1) 



Thus for the particular network topology of Figure [T] decoding 
may actually be achieved in O(K). 




Fig. 1. A simple network 



The obvious fundamental question is how to reduce de- 
coding complexity, exploiting the topology of an arbitrary 
network. We propose to use network topology to construct 
a factor graph for the network code, resulting in a simple 
message-passing decoding algorithm. Despite the simplicity of 
this idea, to the best of our knowledge, this paper is the first 
systematic development of these methods for linear network 
codes. Iterative decoding in a network coding setting has 
been briefly considered in [8], however this work was for a 
specific network topology, and did not develop the idea more 
generally. Iterative decoding has also been considered in [9] in 
which the decoding structure was imposed by a Low Density 
Parity Check degree distribution instead of the topology of the 
network. 

Section [II] provides the necessary background on factor 
graphs, and introduces a new "support passing" algorithm. 
Roughly speaking, this is the sum-product algorithm for sce- 
narios where we don't care about the "probabilities" them- 
selves, only whether they are non-zero. In Section [III] we 
describe how to construct factor graphs from network codes, 
and show that the messages utilized by the support-passing 



algorithm are cosets of linear subspaces. Section IV describes 



how to ensure convergence to the desired solution via cluster- 
ing operations. 

II. Factor graphs and message passing 

Factor graphs [10] are a graphical representation of the 
factorization of a global function into a product of local 
functions. Factor graphs (and related graphical structures such 
as Bayesian networks and Tanner graphs) and the attendant 



message passing algorithms for marginalization of the global 
function have found widespread application [11]. In particular, 
these methods are a key ingredient in modern coding schemes 
such as low-density parity check codes [12]. 

Definition 1 (Factor graph): Consider a global function g 
that factors into a product of local functions 
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where <x, is a subset of {x\, . . . , x n }. A factor graph for 
g is a bipartite graph (V,.F, £) with variable nodes V = 
{Xi, . . . , X n }, factor nodes T = {<fij : j € J} and edges 

£ ^p,i J )eVxf:x,e4 
Direct brute-force computation of the marginals of a global 
function is computationally expensive. However if the global 
function factors in a simple manner, computing marginals 
can be simplified using the well-known sum-product algo- 
rithm [10], briefly reviewed below. Recall that the sum and 
product operations involved are those of the relevant semi- 
ring [13]. 

Definition 2 (Sum-Product Algorithm): At step k, mes- 
sages l^x^^ix) are sent from variable nodes X to a factor 
nodes tf>, and messages v^ ) _ fX (x) are sent from factor nodes 
to a variable nodes X. The message updating rules are 

/4 + i (x)= II 4^x(x) (2) 

4*x( x )= E h(x,x*:X*en(<P)\X) (3) 

x*:X*en(0)\X 

where 

h(x,x* :X* en(4>)\X) = 

:X* Gn(0)\X) JJ 

X*en(<f>)\X 

A well-known fundamental result (see e.g. [10]) regarding the 
optimality of the sum-product algorithm is restated as follows. 

Theorem 1 (Factor tree): If the factor graph is a tree, then 
after a finite number of updates, all messages will remain 
unchanged. Furthermore, for any x a , 

g(xi,...,x n )= Yl 4^x a M (4) 

Theorem [T] shows that when the underlying factor graph is a 
tree, the sum-product algorithm yields the desired marginals 
by Q. If the underlying graph is not a tree, then the sum- 
product algorithm may not converge, and if it does converge, 
it may converge to an incorrect solution. 

As we shall see in Section [III] there are meaningful scenar- 
ios where we are not interested in the value of the marginals, 
but rather their support. In such cases, we can simplify the 
sum-product algorithm. 

Theorem 2 (Support passing algorithm): Assume all local 
functions <pj(xi : i £ aj) are nonnegative. Let Hx^d> 



and v%_>x ^ e messa g es being passed by the sum-product 
algorithm. Then 
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A ( rt+h )= |J A S n(0) \ X)) (6) 
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Proof: By direct verification. ■ 
According to Theorem [2] if only the supports of the marginals 
are required, then one can simplify the sum-product algorithm 
by passing only the supports of the messages. Furthermore, 
the message update rules are given by |5} and |6]). 

Theorem 3 (Convergence of the support passing algorithm): 
Let g be the global function and assume the support passing 
algorithm is used. Initialize A ^jj. x _^^j = A ^jj^x) = ^ 
for all variable node X, where X is the alphabet of variable 
X. Then 

1) A C A ( M ^) and A (^) C A (u^ x ) 
for all /c. Hence, after a finite number of iterations, 
supports of message will remain unchanged; 

2 ) A (j2x*-.x*^x 9(xi, ■ ■ • , in)) is a subset of A 

and a(^u x ). 

Proof: A direct consequence of <|3j and ([6j. ■ 

Theorem [3] guarantees convergence of the support passing 
algorithm, even for graphs with cycles. If the factor graph is a 
tree, the algorithm converges to the support of the marginals. 
However, if the factor graph contains cycles, it still converges, 
but to a limit that contains the support of the marginals. Thus 
the presence of cycles can cause convergence to an undesired 
solution (although it will "contain" the desired solution). 

III. Factor Graphs for Network Codes 

Let Y s be the symbol generated by the source s £ K, and Yj 
be the symbol transmitted on link I £ C. For each link I, the 
node tail(Z) receives symbols generated from the sources or 
transmitted from its neighboring nodes. Considering stochastic 
encoding, the symbol transmitted on link I £ C, namely yi, 
is randomly selected according to a conditional distribution 
Ci(yi\y a ,a £ hi(Z))- Deterministic codes are obtained via 
degenerate C\. Note that this set-up also permits modeling 
of noisy links (incorporating random errors into the C/). 

The probability distribution of (Y S) Yi, s £ /C, I £ C) is 



Pr(y : a € £ U /C) oc JJC,( W | Uin(l))- 



(7) 



as y* for j £ J . Then it is straightforward to prove that 

Pr(y a :aeCL)IC\y*,ieJ) 



U.Ci(y, 
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With error-free links and a uniquely decodeable code (linear or 
non-linear), it can easily be shown that decoding is equivalent 
to finding the support of the {Y s ,s £ K,} marginal of |[S). 
In more general settings, it may be desired to compute the 
marginal posterior probabilities of the source variables. 

In the following, we will provide a mechanical approach 
to represent the global function ([8| with a factor graph. The 
method works for any network codes. However, as we shall 
show in the next example, the obtained factor graph may not 
be the simplest possible. 

Definition 3 (Network Code Factor Graph): Given the net- 
work code defined by stochastic local encoding functions Ci, 
define a factor graph with variable nodes V = {Y s : s £ IC,Yi : 
I £ £} and factor nodes T = {fa : I £ £} U {tpj : j £ J}. 
Each factor node is associated with the local function 
Ci(yi\yi n m), and is connected to variable nodes Yi and Yi 
where i £ in(7). Factor node tpj is associated with the local 
function 5(yj — y*), and is connected only to the variable node 

The topology of the above factor graph depends only on the 
network topology but not the particular codes being used. As 
a result, the obtained factor graph may not be in its simplest 
possible form in general. For example, if some local functions 
(pi can be further factorized into a simpler form, one may 
be able to further simplify the factor graph. Furthermore, in 
the context of network coding, only the variables generated 
by sources are of interest. In that case, it may be possible to 
prune the factor graph without affecting the decoding process. 

Example 2: Consider the network shown in Figure 2(a) 
The symbols generated by the source are y\ and y 2 - The 
receiver t can observe symbols j/4 and 7/5. The factor graph 
for this code (with respect to the given receiver) is depicted 
in Figure |2(b)| 

Depending on the specific choices of encoding functions, 
this factor graph may be simplified. Suppose 1/3 is a function 
of y\. Then one can simplify the factor graph by removing the 
link between the factor node (p 3 and the variable node Y 2 . 
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(a) Network (b) Factor graph 

Fig. 2. A relay network 



Consider a receiver which observes incoming symbols Yj 



Let us now consider the special, but important case of 
deterministic linear network codes. All y s for s S JC are 
independent and uniformly distributed over F" and for each 
link I G £, 

Vl = ^2 Cl,eVe 
eGin(Z) 

where the local encoding kernel coefficients Q, e are fixed and 
known. Hence, 



Ci(yi\y in (i)) 



1 if Vl = Eeein(i) °l^Ve 

otherwise 
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Theorem 4 (Deterministic linear network codes): Assume 
that the support passing algorithm is used. Suppose that 
Y.- L = y* is the actual symbol being transmitted or generated. 
Then 

1) Py^^ and v k ^} Y are cons tant over their supports. There- 
fore, the support passing algorithm and the sum-product 
algorithm are equivalent in this case; 

2) A ( Py^i> ) is a coset of the form y 



Wy^j. is a vector subspace; 



Wyti where 



Y^<p 
3) Similarly, A 



is also a coset of the form y* 



Wy^ 6 wriere Wy^^ is also a vector subspace. 



Y^<p 
Furthermore, 
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(11) 



(12) 



where (Wy*^ '■ Y* g n(0) \ Yj is defined as the minimal 

subspace containing W Y »^^ for all Y* £ n(0) \ Y. 

Theorem |4] shows that when deterministic linear codes 
are used, the support passing algorithm can be implemented 
by "updating" subspaces according to ( fTT| and fl2) . In the 
case that the underlying factor graph is acyclic and the 
network code is uniquely decodeable (i.e. invertible global 
linear transform), this subspace update algorithm is guaranteed 
to converge to the correct solution. 



Example 3: Consider a butterfly network in Figure 3(a) 



The sources are y\ and j/2> and the sinks are denoted by open 
circles. A factor graph for the butterfly network is given in 
Figure |3(a)| Note that, although the butterfly network itself has 
an undirected cycle, the corresponding factor graph, depicted 



in Figure 3(b) is cycle-free. Therefore, message passing and 
support-passing algorithms are exact. 

Suppose sink t\ observes y$ and ys and aims to reconstruct 
yi and y^. Then some of the links in the factor graph Figure 
|3(b)| can be removed without affecting decoding at t\. This 
results in the simplified factor graph Figure 3(c) 
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(a) Butterfly network 




(b) Factor graph 
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(c) Factor graph (after pruning) 
Fig. 3. Butterfly network and its factor graphs 

IV. Dealing with Cycles 

In the previous section, we proposed a simplified sum- 
product algorithm which passes only the supports of messages. 
It was proved that the algorithm always converges, and that 
is equivalent to the sum-product algorithm in the case of 
deterministic linear network codes. As a result, when the 
underlying factor graph is a tree, the proposed support-passing 
algorithm ensures that the support of marginals can be found. 

Unfortunately, when the factor graph has cycles, the 
sum-product algorithm does not always converge. Although 
support-passing algorithm converges, it may not converge to 
the supports of the desired marginals. 

To avoid cycles, one may transform a factor graph with 
cycles into one with no cycle. For example, one common 
transformation technique is clustering [10] as demonstrated 
in the following example. 



Example 4: Consider the network depicted in Figure 4(a) 



which is a simplified version of the network shown in Figure 
[T] From the network, we can construct a factor graph in 
Figure 4(b) according to Definition [3] By clustering function 
nodes together, we can transform the factor graph into a cycle 
free one as shown in Figure 4(c)| In fact, running the sum- 



product algorithm over the cycle free factor graph is essentially 
equivalent to decoding by ([TJ. 




(a) Original network 




(b) Cyclic factor graph (c) Factor graph after clustering 

Fig. 4. Factor graph transformation 



Recall that every link is associated with a factor node. As 
a general rule, the first step of clustering is to cluster those 
factor nodes associated with links originating from the same 
node. 

V. Conclusion 

Decoding of network codes is traditionally achieved by 
solving a system of linear equations. Using this approach, the 
decoding complexity of the network is essentially independent 
of the underlying topology. This paper shows that if we exploit 
our knowledge of the topology, a more efficient decoding 
algorithm may be obtained. In some extreme examples, we 
showed that the reduction in decoding complexity can be huge. 
This paper prompts a new direction in network code design: 
how to choose a network subgraph such that the resulting 
network code admits an efficient decoding algorithm. 

As the first step towards the goal, we propose the use 
of message passing algorithm as a decoding strategy. For a 
given network code, we give algorithms to construct a factor 
graph on which message passing algorithms (such as the 
sum-product algorithm) are performed. We proved that when 
network links are noiseless, the support passing algorithm (a 
simplified version of the sum-product algorithm) suffices. We 
showed that the support-passing algorithm is exact when the 
underlying factor graph is acyclic and always converges to 
a limit which contains the desired solutions, even when the 
factor graph is cyclic. Finally, we discussed the use of some 
graph augmentation techniques to transform a cyclic factor 



graph to an acyclic one so that the support passing algorithm 
is exact. 
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